Efficient DNN Model for Word Lip-Reading

This paper studies various deep learning models for word-level lip-reading technology, one of the tasks in supervised video classification. Several public datasets have been published research field. However, few investigated techniques using multiple datasets. evaluates four publicly available datasets, namely Lip Reading Wild (LRW), OuluVS, CUAVE, and Speech Scene by Smart Device (SSSD), which are representative this LRW is large-scale targets 500 English words released 2016. Initially, recognition accuracy was 66.1%, but many groups working on it. The current state art (SOTA) has achieved 94.1% 3D-Conv + ResNet18 {DC-TCN, MS-TCN, BGRU} knowledge distillation word boundary. Regarding SOTA model, paper, we combine existing such as ResNet, WideResNet, EfficientNet, Transformer, ViT, ViViT, investigate effective six with modified feature extractors classifiers. Through experiments, show that similar model structures extraction MS-TCN inference valid different scales.